Mixed Risk-Neutral/Minimax Control of Markov Decision Processes
نویسندگان
چکیده
This paper introduces a formulation of the mixed risk-neutral/minimax control problem for Markov Decision Processes (MDPs). Drawing on results from risk-neutral control and minimax control, we derive an information state process and dynamic programming equations for the value function. Furthermore, we develop a methodology to synthesize an optimal control law on the nite horizon, and a near-optimal control law on the innnite horizon, both for the fully observed and partially observed cases. We compare the mixed risk-neutral/minimax approach to the risk-sensitive control of MDPs.
منابع مشابه
Mixed risk-neutral/minimax control of discrete-time, finite-state Markov decision processes
This paper addresses the control design problem for discrete-time, nite-state Markov Decision Processes (MDPs), when both risk-neutral and minimax objectives are of interest. We introduce the mixed risk-neutral/minimax objective, and utilize results from risk-neutral and minimax control to derive an information state process and dynamic programming equations for the value function. We synthesiz...
متن کاملRisk - Sensitive , Minimax , and Mixed Risk - Neutral / Minimax Control of Markov Decision Processes
This paper analyzes a connection between risk-sensitive and minimax criteria for discrete-time, nite-state Markov Decision Processes (MDPs). We synthesize optimal policies with respect to both criteria, both for nite horizon and discounted in nite horizon problems. A generalized decision-making framework is introduced, leading to stationary risk-sensitive and minimax optimal policies on the in ...
متن کاملRisk-sensitive and minimax control of discrete-time, finite-state Markov decision processes
This paper analyzes a connection between risk-sensitive and minimax criteria for discrete-time, nite-states Markov Decision Processes (MDPs). We synthesize optimal policies with respect to both criteria, both for nite horizon and discounted in nite horizon problem. A generalized decision-making framework is introduced, which includes as special cases a number of approaches that have been consid...
متن کاملA Framework for Mixed Estimation of Hidden Markov Models
In this paper, we present a framework for a mixed estimation scheme for hidden Markov models (HMM). A robust estimation scheme is first presented using the minimax method that minimizes a worst case cost for HMMs with bounded uncertainties. Then we present a mixed estimation scheme that minimizes a risk-neutral cost with a constraint on the worst-case cost. Some simulation results are also pres...
متن کاملOn Minimax Optimal Offline Policy Evaluation
This paper studies the off-policy evaluation problem, where one aims to estimate the value of a target policy based on a sample of observations collected by another policy. We first consider the multi-armed bandit case, establish a minimax risk lower bound, and analyze the risk of two standard estimators. It is shown, and verified in simulation, that one is minimax optimal up to a constant, whi...
متن کامل